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ABSTRACT 

As the use of National Science Foundation (NSF) -sponsored, 
reform- oriented mathematics curricula has become more prevalent across the 
U.S., an increasing number of researchers are attempting to study the 
"impact” of reform. In particular, mathematics educators are interested in 
determining whether reforms are having the desired effects on students, 
particularly with respect to the learning of mathematical content and the 
improvement of attitudes about mathematics. In this effort, researchers have 
used a variety of methods, and have looked at a variety of variables, in 
order to assess the impact of reform. In many cases, such research assesses 
reform by looking closely at students* scores on tests or their strategies 
for solving certain kinds of problems. For example, Riordan & Noyce (2001) 
assessed reform’s impact by comparing students’ scores on standardized 
achievement tests. Other researchers have used structured interviews, 
classroom observations, and more interpretive or ethnographic methods to 
assess the impact of reform (e.g., Boaler, 1997). Both of these methodologies 
are useful in assessing the impact that reform mathematics curricula are 
having on students. An alternative evaluation of the impact of reform that 
has not been as widely used is through the use of survey instruments. Surveys 
have been widely and reliably used to assess students’ motivation (Pintrich, 
Smith, Garcia, & McKeachie, 1993) , beliefs and attitudes (Kenney & Silver, 
1997), and interest (Ksller, Baumert, & Schnabel, 2001). We propose to add to 
this literature by using a survey to study the impact of reform on students’ 
conceptions of mathematics. (Author) 
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Introduction and Perspective 

As the use of NSF-sponsored, reform-oriented mathematics curricula has become 
more prevalent across the US, an increasing number of researchers are attempting to 
study the impact of reform. In particular, mathematics educators are interested in 
determining whether reforms are having the desired effects on students, particularly 
with respect to the learning of mathematical content and the improvement of attitudes 
about mathematics. In this effort, researchers have used a variety of methods, and have 
looked at a variety of variables, in order to assess the impact of reform. In many cases, 
such research assesses reform by looking closely at students’ scores on tests or their 
strategies for solving certain kinds of problems. For example, Riordan & Noyce (2001) 
assessed reform’s impact by comparing students’ scores on standardized achievement 
tests. Other researchers have used structured interviews, classroom observations, and 
more interpretive or ethnographic methods to assess the impact of reform (e.g., Boaler, 
1997). Both of these methodologies are useful in assessing the impact that reform 
mathematics curricula are having on students. An alternative evaluation of the impact 
of reform that has not been as widely used is through the use of survey instruments. 
Surveys have been widely and reliably used to assess students’ motivation (Pintrich, 
Smith, Garcia, & McKeachie, 1993), beliefs and attitudes (Kenney & Silver, 1997), 
and interest (Roller, Baumert, & Schnabel, 2001). We propose to add to this literature 
by using a survey to study the impact of reform on students’ conceptions of mathemat- 
ics. 
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Background 

Few survey measures have been developed for the purpose of studying secondary 
students’ conceptions of mathematics. One such measure is the Conceptions of Math- 
ematics Inventory [CMI] (Grouws, 1994). The CMI is intended to assess students’ 
beliefs or conceptions about mathematics. Its 56 questions asked students whether they 
agreed or disagreed with certain statements about what it means to do, learn, and think 
about mathematics. The survey questions fall into seven categories (the composition 
of mathematical knowledge, the structure of mathematical knowledge, the status of 
mathematical knowledge, doing mathematics, validating ideas in mathematics, learn- 
ing mathematics, and the usefulness of mathematics), each of which assesses a differ- 
ent aspect of students beliefs toward math. Students’ responses for each question were 
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between 1 and 6, with “1” expressing strong agreement and “6” expressing strong 
disagreement. A student who mostly agrees with all questions would seem to have an 
attitude that the survey designers felt was consistent with the aims of recent reform 
documents. Such a student would view mathematics as being composed of a useful, 
coherent, and dynamic system of concepts and ideas, where learning is accomplished 
by sense-making and authority is found through logical thought. A student who mostly 
disagrees with statements on the CMI would find mathematics an irrelevant, unchang- 
ing collection of isolated facts and procedures, handed down from a book or teacher, 
that must be memorized; this is a view that the CMI designers consider more typical 
of traditional math curricula. 

Grouws and colleagues (Grouws, Howald, & Colangelo, 1996) gave the CMI 
originally to 163 9th, 10th, and 11th graders in traditional mathematics classes. Their 
intent was to determine whether students in advanced mathematics classes had dif- 
ferent conceptions of the discipline than students in “regular” track. Although the 
questions on the CMI are intended to indicate whether students’ conceptions are 
consistent with the goals of reform, the CMI has never been administered to a large 
group of students in reform mathematics classes to “validate” its effectiveness. In 
other words, although it may seem obvious that students with extensive experience 
in reform mathematics would have more reform-oriented conceptions than those with 
extensive experience in traditional mathematics, we made an effort to empirically 
explore whether students in reform and traditional settings would respond differently 
on a validated survey instrument. 

In this paper, we describe our attempt to validate the CMI. In addition to provid- 
ing evidence of the effectiveness of the instrument, such a validation also allows the 
CMI to be used as a measure of the impact of reform, in that it establishes “baseline” 
values for reform and traditional students’ conceptions. 

Method 

In conversations with the authors of the CMI (Grouws, personal communication, 
October 5, 2001), we became convinced that all of the 163 original respondents to 
the CMI (Grouws et al., 1996) came from traditional backgrounds in mathematics. In 
other words, although no data were collected about the schools, courses, or instruction 
experienced by students in Grouws and colleagues’ original sample, we are confident 
that no student in this sample had any recent experience with NSF-funded reform- 
oriented curricula, and thus experienced something of a default approach to teaching 
mathematics that was more traditional than reform-oriented. 

For the present study, and as a contrast, we sought to assess a sample of students 
who had as “pure” of a reform experience in mathematics as we could find. As part 
of our work in a number of teacher professional development and curriculum writing 
projects in Michigan, we developed relationships with several schools with exemplary 
enactments of reform curricula, particularly at the middle school level. Through these 
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connections, we recruited 134 9th grade students from a high school in Michigan to 
complete the CMI, early in their 9th grade year. All students completed at least three 
years of reform-oriented instruction (in 6th, 7th, and 8th grades) in a middle school 
whose curriculum (Connected Mathematics Project, Lappan, Fey, Fitzgerald, Friel, & 
Phillips, 1997) and instruction we were quite familiar with and that we are confident 
was an extremely well-enacted version of reform. Students were administered the 
CMI in their regular mathematics classes by their teacher. 

Results 

To summarize our main results, we found that students from a reform background 
responded differently to the items on the CMI than did students from a traditional 
background. In particular, students’ responses in the reform setting were more 
aligned with reform- oriented ideas on the scales of the CMI than traditional students’ 
responses. Table 1 shows students’ mean scores on the six scales of the CMI, for both 
groups of students. Recall that all items used a six-point scale, and that the lower the 
number, the more reform-oriented the response. The differences on each scale are 
statistically significant, p < S)\. 



Table 1. Participants’ mean scores (standard deviations) on CMI scales 
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Discussion 

Our results provide evidence that the CMI is a useful and valid instrument for 
assessing the impact of reform. We found that after experiencing several years of 
exemplary instruction in a reform curricula, students do develop conceptions of math- 
ematics that are aligned with NCTM reform documents. 

Assessing the impact of reform is a complex endeavor, requiring the investigation 
of many aspects of students’ experiences and using multiple methods. We believe that 
assessing students’ beliefs about mathematics is vital to this effort. 
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